Direct-gradient optical image correlation apparatus

ABSTRACT

An electronic system for achieving correlation and matching information between two similar optical displays without mechanically or electronically nutating or moving one display relative to the other. The system developes from each display a series of voltage signals which are highly informative of the display elements. The voltage signals from one display are summed according to a schedule established by the other display to obtain a differential signal which is scaled to determine the direction and amount of position or offset error between the two displays. One display may be a live image of a target area while the second display is a reference image of the same area.

United States Patent [1 1 Diamantides 11] 3,748,042 [45] July 24, 1973DIRECT-GRADIENT OPTICAL IMAGE CORRELATION APPARATUS [21] Appl. No.:148,359

Related US. Application Data [63] Continuation-in-part of Ser. No.592,532, Nov. 7,

1966, Pat. No. 3,609,762.

[52] US. Cl. 356/163 [51] Int. Cl. ,G0lb 11/00 [58] Field of Search356/2, 163, 164, 356/202, 203

[56] I References Cited UNITED STATES PATENTS 3,246,560 4/1966 Birbaumet al. 356 2 Primary Examiner-William L. Sikes- Attorney-J. G. Pete andL. A. Germain 57 ABSTRACT An electronic system for achieving correlationand matching information between two similar optical displays withoutmechanically or electronically nutating or moving one display relativeto the other. The system developes from each display a series of voltagesignals which are highly informative of the display elements. Thevoltage signals from one display are summed according to a scheduleestablished by the other display to obtain a differential signal whichis scaled to determine the direction and amount of position or offseterror between the two displays, One display may bea live image of atarget area while the second display is a reference image of the samearea.

8 Claims, 11 Drawing Figures YHRESHOLD SET CIRCUIT I 60'; I TRANSVERSE64 DEFLECTION GENERATOR LONGITUDINAL DEFLECTION GENERATOR MULTIVIBRATORINTEGRATOR PATENIED 24375 SHEET 1 HF 2'4 24a 24b 24c24d BA u FIG-4 hmFlG.-2e t F l G.- 2 1 o 20; 20c 20. hm

22 22a 22b 22c 22d FIG-2c Law 241: 24c I 1 r 24 l L 24A INVENTORNICHOLAS D. DIAMANTIDES WWW ATTORNEYS DIRECT-GRADIENT OPTICAL IMAGECORRELATION APPARATUS This application is a continuation-in-partapplication of Ser. No. 592,532, filed Nov. 7, l966 for DIRECTGRADIENT-CORRELATION APPARATUS, now U.S. Pat. No. 3,609,762, grantedSept. 28,1971.

The above mentioned patent discloses a system for matching orcorrelating a radar image with a reference image. The present inventionrelates to the adaptation of the techniques and system of the patent tothe correlating of an optical image and a reference image.

The primary object of the present invention is the provision of adirect-gradient optical image correlation apparatus which may achievematching between a live optical image and a reference image without theneed of nutation while achieving rapid and accurate image matching.

It is also an object of the present invention to provide adirect-gradient optical image correlation system which concentrates onthose elements of the live image having high informational content whiledisregarding the elements of the live image which are subject tovariation such as seasonal or diurnal changes. The above and otherobjects of the invention which will become apparent in the followingdetailed description are achieved by providing correlation apparatuswhich includes means for scanning the live image and forming a series ofvoltage pulses corresponding to the boundary lines between regions ofhigh and low illumination of the live image, means to represent andstore the reference. image as a plurality of voltage pulses, and meansto compare the live image pulse signals and the reference signal todetermine the direction and amount of position or offset error betweenthe live image and the reference image.

For a morecomplete understanding of the invention and the objects andadvantages thereof reference should be had to the following detaileddescription and the accompanying drawings wherein there is shown apreferred embodiment of the invention.

In the drawing:

FIG. la is a pictorial view of a liveiimage;

FIGS. lb and 1c are diagramatic views of the live image of FIG Ia andillustrate the theory by which the voltage signal representative of thelive image is generated;

FIGS. 2a 2e are diagrams illustrating the development of the voltagesignal forone line of scan across the image of FIG. 1a;

FIG. 3 is a highly schematic showing of the directgradient optical imagecorrelation apparatus of the present invention;

FIG. 4 is a schematic showing of an optical imaging system; and

FIG. Sis a pictorial view, partially in section, of a image-dissectorphotomultiplier employed in the system of FIG. 3.

Considering the target or live image 10, if this live image is projectedby a lens or optical imaging system 12 onto a plane 14, a real image 16of the live image is formed on the plane. This image 16 may be describedby the function [(x, y) where I is the measure of light intensity of theimage at the point (x, y). As will be described in more detail below,the function [(x, y) may, by the use of a dissecting photomultiplier, beconverted into a voltage signal V(x, y) which is directly proportionalto l(x, y). FIG. 2a illustrates the value of V(x, y) for one value of y,that of the line 18 of FIG. la. illustrates the value of V(x,y) for onevalue of that of the line 18 of FIG. a. It will be noted that the liveimage 10 has light areas 20, 20b, 20d and 20f, which are represented aspeaks in the graph of the function V since these regions are the regionsof highest illumination of the real function 16. The real image also hasdark or shaded regions 20a, 20c, and 20e which are represented asminimums in the graph. It should be noted that the slope of the curve ofFIG. 2a is greatest at the boundaries between areas of high and lowillumination. This fact is utilized by the present invention to providea reliable correlation technique. The maximums and minimums of thederivative V,,(x,y) of V (x,y) correspond to these points of greatestchange and, hence, to the boundaries between adjacent light and darkregions of the live image, as shown in FIG. 2b.

If the derivative is then squared, the resulting curve, as

shown in FIG. 2c, will indicate the boundaries between successive lightand dark or dark and. light areas as peaks. 1

For a variety of reasons it is expedient to use only the strongestportions of the derivative of V in both the positive and negativepolarities, and to reject the remainder by means of a threshold. Quiteobviously such a threshold, th, cannot be absolute and universal for allcases; instead it should be dependent on the live image 10. Thethreshold can be defined as follows:

th=krr where k constant and 0' variance of V, (x,y). A

function V t) canthen be defined as follows:

,v (t) A if v. (x,y) m, 0 0 otherwise.

Likewise,

' ,v'o A if 0, (39y) th,, 0'

' 0 otherwise. I

It is obvious that the signal ,V (I) will be a train of positive pulsescorresponding to the image locations of sharp boundary lines regardlessof polarity. Hence, the process so far is in essence a selectiveedge-enhancing and accomplishes two important functions: (1) retains thesignal only at image locations of high informational content, and (2)disregards polarity. The first function reduces the quantity ofinformational data' which must be stored and shortens the time.requir'edto correlate the live reference images. The latter functioneliminates the problem of polarity reversals of local image brightness.Such reversals may be .due to seasonal changes of scenery or to'diurnalchanges of illumination. A lake, for instance, may appear either brightor dark depending on whether it reflects thesun toward or away from thesensor.

FIG. lb illustrates the live image 10 reduced to the boundary line form.This figure illustrates the substantial reduction in information datawhich need be considered by the correlation apparatus.

In order to solidify picture elements which are strongestinformationally while rejecting the rest and to relieve the opticpicture matching from the bane of high sensitivity to scale and azimuthorientation discrepancies between the reference and live images, widthe'nhancing of the pulses of V (I) is employed. This is illustrated inFIG. 2e where the boundary points are indicated as pulses of a uniformwidth. The equivalent picture of the live image 10 is illustrated inFIG. 1c.

FIGS. 3, 4, and illustrate one embodiment of the apparatus which may beemployed to achieve the objects of the invention. The real image 16formed by the imaging system 14 on the photocathode 32 of the imagedissector produces an electron image which is current-density modulatedaccording to the optical input matter. This electron image isaccelerated and projected through a dissecting aperture 34, on anelectron multiplier 36 and from there to an anode 38. Deflection coils60 and 62, serve to controllably scan the electron image across thedissecting aperture 34; by the operation of the deflection coils a scancan be taken along a line in either the x or y directions across thereal image 16. For example, the scan may be along the line 18 of FIG. laproducing the output current at the anode 38 which corresponds to FIG.2a. The image dissector may be a commercially available unit such as avidissector tube manufactured by ITT which employs magnetic focusing andmagnetic deflection and an electron-emissive metal mesh in closeproximity to the photocathode, followed by an electric-field-free focusand deflection drift space. Because the live image may change rapidly astorage grid may be employed in the image-dissector to retain the realimage at a particular instant and hold this image until an erasingaction is taken. This version of a tube is called storage imagedissector and is available commercially also. This allows the picture tobe kept invarient for a time period long enough to perform the signalprocessing of the present invention, as discussed below. The voltageoutput of the image disSector 30 is differentiated by the differentiator42. The differentiated signal is then squared by the multiplier 44. Fromthis point the signal is subjected to two different operations takingplace sequentially and initiated by the two-position electronic switch70. The first operation takes place where the switch is in position 46achanneling the signal V, to the circuit boundary that comprises theintegrator 48 and the threshold setter 50. In the integrator, the signalV, is averaged over the whole picture, thus producing the variance 0-. Aportion A of ais held in storage at the threshold setter 50 and appliedto the threshold gate or comparator 52. Once this is done the secondoperation is initiated by the switch 70 which switches to the position46b. As before, the transversal (horizontal) and longitudinal (vertical)deflection coils acting cooperatively scan the image in the x directionline by line regenerating the signal V ()nly the strong peaks of thelatter are allowed by the threshold gate 52 to proceed into the singleshot multivibrator 54 triggering the latter into producing a train ofstandard pulses as depicted by FIG. 2e. This pulse train is symbolizedby in FIG. 3 (hm meaning the m-th horizontal scan line), and isintegrated over each scan line by the integrator 56, thus producing them-th point of the vertical map M,,:

Hence, M (m A y) represents the total number of strong boundariescrossed by the m-th scan line.

As the longitudinal (vertical) deflection coil changes the value of m,one by one the points of the linear map M,, in the y direction aregenerated. This is exactly the map form required of maps suitable fordirect-gradient map matching.

According to the direct-gradient technique, the cor responding referencemap M,,* may be prepared by (a) applying the foregoing operations to areconnaissance photo for generating a map M similar to the live map Mand (b) subjecting M to the average-level-crossing criterion of thedirect-gradient technique in order to produce the reference pulse trainM,,*.

The technique and apparatus for performing the correlation between thelive map and the reference map is illustrated and described in theabove-identified patent. Reference should be had to said patent andparticularly to the embodiment illustrated in FIG. 4.

Since the reference maps are pulse trains in which pulse spacing is theinformation carrier, and since the number of pulses in each train isrelatively small, in the order of 15 pulse pairs or less, the referencemap is storable in a small digital memory thus obviating the need forcumbersome electro-optical components such as photographic film,film-movement mechanisms, and projectors as well as all the necessaryproblems of alignment, weight and malfunction inherent in such systems.Further, the reference map may be changed remotely within milliseconds,by any form of data link with the comparator apparatus. Since thereference map is digital in nature it is highly immune from noise duringread-in or read-out operations.

The operation of the system is controlled by a clock 58 which governsthe transverse and longitudinal deflection generators 60 and 62,respectively for feeding the corresponding deflection coils in the imagedissector 30. The clock 58 also controls an alternator 64 which operatesa switch 66 for switching the operations between the x and y axes of theimage and the switch 70 for switching between threshold determinationand comparator operations.

The method of the present inventiOn frees the matching from the effectsof clouds that may be present within the live map area. This happensbecause the de.- rivative signals are quite low over cloud areas, and assuch, are eliminated by the threshold. Only at the cloud edges willthese signals possibly be substantial; however, by not being present inthe reference, they count primarily as a low frequency additive noise,toward which the direct-gradient mapping exhibits considerableinsensitivity. g l

The fact that the picture is scanned along lines parallel to either thex-or the y axis offers a decisive tool for undoing the effects of largediscrepancies due to large differences in attitude and orientationbetween the points at which the reference and live pictures were taken.It is obvious that an appropriate computer program can easily generatethe shape of the line into which a given straight scan line should betransformed to undo such effects. Matching is therefore possible betweena reference obtained vertically and a live picture obtained from ashallow trajectory or vice-versa.

It will be understood that while only the best known embodiments of theinvention have been illustrated and described in detail herein, theinvention is not so limited. Reference should therefore be had to theappended claims in determining the true scope of the invention.

What is claimed is:

1. Apparatus for generating a map signal from an optical image for useas an input in a gradient-based correlator, comprising:

means for generating a signal proportional to the intensity of theoptical image along at least one scan line traversing the image; firstcircuit means receiving the signal from the signal generating means andproducing a signal equivalent to the arithmetically squared rate ofchange of the signal from the signal generating means;

second circuit means for receiving the signal produced by the firstcircuit means and creating therefrom a predetermined signal level;

third circuit means for receiving the signal produced by the firstcircuit means and the predetermined signal level produced by the secondcircuit means and producing an output signal when the signal produced bythe first circuit means exceeds the predetermined signal level createdby the second circuit means; and

a first integrator for integrating the output signal from the thirdcircuit means for each scan line to produce the map signal.

2. Apparatus according to claim 1 wherein the first circuit meanscomprises a differentiator receiving the signal produced by the signalgenerating means, and a first multiplier for squaring the output of thedifferentiator.

3. Apparatus according to claim 2 wherein the second circuit meanscomprises a second integrator receiving the output of the firstmultiplier, a second multiplier for multiplying the second integratoroutput by a constant, and storage means for retaining the output of thesecond multiplier and for furnishing said output to the third circuitmeans.

4. Apparatus according to claim 3 wherein the third circuit meanscomprises a comparator in series connection with a multivibrator.

5. Apparatus according to claim 1 wherein the means for generating thesignal comprises an image dissector having an optical input.

6. Apparatus according to claim 5 further including means to project areal image of a live object to the optical input of the image dissector.

7. Apparatus for generating a signal representative of an optical image,comprising:

means for scanning the optical image to produce a signal proportional tothe intensity of the optical image along at least one line of scantraversing the image;

first circuit means receiving the signal from the scanning means andproducing an output pulse indcative of the squared rate of change of thesignal from the scanning means;

second circuit means responsive to the first circuit.

means for establishing a predetermined signal level;

third circuit means responsive to the first and second circuit means forproducing an output signal when the signal from the first circuit meansexceeds the predetermined signal level of the second circuit means; and

an integrating circuit for integrating the output pulse signal for eachline of scan to produce the representative signal.

-8. Apparatus according to claim 7 wherein the scanning means scansalong at least two non-parallel lines each traversing the image toproduce at least two signals, and further including means controllingthe scanning means and the first, second, and third circuit meanswhereby the representative signal corresponding to each line of scan isgenerated successively.

1. Apparatus for generating a map signal from an optical image for useas an input in a gradient-based correlator, comprising: means forgenerating a signal proportional to the intensity of the optical imagealong at least one scan line traversing the image; first circuit meansreceiving the signal from the signal generating means and producing asignal equivalent to the arithmetically squared rate of change of thesignal from the signal generating means; second circuit means forreceiving the signal produced by the first circuit means and creatingtherefrom a predetermined signal level; third circuit means forreceiving the signal produced by the first circuit means and thepredetermined signal level produced by the second circuit means andproducing an output signal when the signal produced by the first circuitmeans exceeds the predetermined signal level created by the secondcircuit means; and a first integrator for integratinG the output signalfrom the third circuit means for each scan line to produce the mapsignal.
 2. Apparatus according to claim 1 wherein the first circuitmeans comprises a differentiator receiving the signal produced by thesignal generating means, and a first multiplier for squaring the outputof the differentiator.
 3. Apparatus according to claim 2 wherein thesecond circuit means comprises a second integrator receiving the outputof the first multiplier, a second multiplier for multiplying the secondintegrator output by a constant, and storage means for retaining theoutput of the second multiplier and for furnishing said output to thethird circuit means.
 4. Apparatus according to claim 3 wherein the thirdcircuit means comprises a comparator in series connection with amultivibrator.
 5. Apparatus according to claim 1 wherein the means forgenerating the signal comprises an image dissector having an opticalinput.
 6. Apparatus according to claim 5 further including means toproject a real image of a live object to the optical input of the imagedissector.
 7. Apparatus for generating a signal representative of anoptical image, comprising: means for scanning the optical image toproduce a signal proportional to the intensity of the optical imagealong at least one line of scan traversing the image; first circuitmeans receiving the signal from the scanning means and producing anoutput pulse indcative of the squared rate of change of the signal fromthe scanning means; second circuit means responsive to the first circuitmeans for establishing a predetermined signal level; third circuit meansresponsive to the first and second circuit means for producing an outputsignal when the signal from the first circuit means exceeds thepredetermined signal level of the second circuit means; and anintegrating circuit for integrating the output pulse signal for eachline of scan to produce the representative signal.
 8. Apparatusaccording to claim 7 wherein the scanning means scans along at least twonon-parallel lines each traversing the image to produce at least twosignals, and further including means controlling the scanning means andthe first, second, and third circuit means whereby the representativesignal corresponding to each line of scan is generated successively.